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When important decisions are made based on test scores, it is critical to avoid bias, 
which may unfairly influence examinees' scores. Bias is the presence of some 
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characteristic of an item that results in differential performance for individuals of the 
same ability but from different ethnic, sex, cultural, or religious groups. 

This digest introduces three issues to consider when evaluating items for bias-fairness, 
bias, and stereotyping. The issues are presented and sample review questions are 
posed. A comprehensive item bias review form based on these principles is listed in the 
references and is available from ERIC/AE. This Digest and the review form are intended 
to help both item writers and reviewers. 

In any bias investigation, the first step is to identify the subgroups of interest. Bias 
reviews and studies generally focus on differential performance for sex, ethnic, cultural, 
and religious groups. In the discussion below, the term designated subgroups of interest 
(DSI) is used to avoid repeating a list of possible subgroups. 

FAIRNESS VS. BIAS 



In preparing an item bias review form, each question can be evaluated from two 
perspectives: Is the item fair? Is the item biased? While the difference may seem trivial, 
some researchers contend that judges cannot detect bias in an item, but can assess an 
item's fairness. Perhaps the best approach is to include both types of questions on the 
review form. (Figure 1 offers a list of questions addressing fairness.) 

DIFFERENT KINDS OF BIAS 



Bias comes in many forms. It can be sex, cultural, ethnic, religious, or class bias. An 
item may be biased if it contains content or language that is differentially familiar to 
subgroups of examinees, or if the item structure or format is differentially difficult for 
subgroups of examinees. An example of content bias against girls would be one in 
which students are asked to compare the weights of several objects, including a 
football. Since girls are less likely to have handled a football, they might find the item 
more difficult than boys, even though they have mastered the concept measured by the 
item (Scheuneman, 1982a). 

An item may be language biased if it uses terms that are not commonly used statewide 
or if it uses terms that have different connotations in different parts of the state. An 
example of language bias against blacks is found in an item in which students were 
asked to identify an object that began with the same sound as "hand." While the correct 
answer was "heart," black students more often chose "car" because, in black slang, a 
car is referred to as a "hog." The black students had mastered the concept but were 
selecting the wrong item because of language differences (Scheuneman, 1982b). 
Questions that might be asked to detect content, language, and item structure and 
format bias are listed in Figure 2. 

STEREOTYPING AND INADEQUATE 
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REPRESENTATION OF MINORITIES 



Stereotyping and inadequate or unfavorable representation of DSI are undesirable 
properties of tests to which judges should be sensitized. Tests should be free of 
material that may be offensive, demeaning, or emotionally charged. While the presence 
of such material may not make the item more difficult for the candidate, it may cause 
him or her to become "turned off," and result in lowered performance. An example of 
emotionally charged material would be an item dealing with the high suicide rate among 
Native Americans. An example of offensive material would be an item that implied the 
inferiority of a certain group, which would be offensive to that group. Terms that are 
generally unacceptable in test items include lower class, housewife, Chinaman, colored 
people, and red man. 

Additional terms to avoid include job designations that end in "man." For example, use 
police officer instead of policeman; firefighter instead of fireman. Other 
recommendations to eliminate stereotyping: 

- Avoid material that is controversial or inflammatory for DSI. 

- Avoid material that is demeaning or offensive to members of DSI. 

- Avoid depicting members of DSI as having stereotypical occupations (i.e., Chinese 
launderer) or in stereotypical situations (i.e., boys as creative and successful, girls 
needing help with problems). 

FIGURE 1 - SAMPLE QUESTIONS ADDRESSING FAIRNESS 

- Does the item give a positive representation of designated subgroups of interest 
(DSI)? 

- Is the test item material balanced in terms of being equally familiar to every DSI? 

- Are members of DSI highly visible and positively portrayed in a wide range of 
traditional and nontraditional roles? 

- Are DSI represented at least in proportion to their incidence in the general population? 

- Are DSI referred to in the same way with respect to the use of first names and titles? 

- Is there an equal balance (across items in the test) of proper names? ethnic groups? 
activities for all groups? roles for both sexes? adult role models (worker, parent)? 
character development? settings? 

- Is there greater opportunity on the part of members of one group to be acquainted with 
the vocabulary? 
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- Is there greater opportunity on the part of members of one group to experience the 
situation or become acquainted with the process presented by the items? 

- Are the members of a DSI portrayed as uniformly having certain aptitudes, interests, 
occupations, or personality traits? 

FIGURE 2 - SAMPLE BIAS QUESTIONS 



CONTENT BIAS- Does the item contain content that is different or unfamiliar to different 
DSI? 

- Will members of DSI get the item correct or incorrect for the wrong reason? 

- Does the content of the item reflect information and/or skills that may not be expected 
to be within the educational background of all examinees? 



LANGUAGE BIAS- Does the item contain words that have different or unfamiliar 
meanings for DSI? 

- Is the item free of difficult vocabulary? 

- Is the item free of group specific language, vocabulary, or reference pronouns? 



ITEM STRUCTURE AND FORMAT BIAS-Are clues included in the item that would 
facilitate the performance of one group over another? 

-Are there any inadequacies or ambiguities in the test instructions, item stem, keyed 
response, or distractors? 

-Does the explanation concerning the nature of the task required to successfully 
complete the item tend to differentially confuse members of DSI? 

RECOMMENDED READING 
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